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Empirical Research on the Software Process 




LIFT experiment 

8 experienced pro- 
grammers designing 
the control structure 
for a set of elevators 
during an intense 2 
hr* session 


Object server exp. 


Field study 


Videotaped team 
meetings from a 7 
mo. effort to design 
and build a tool to 
support object ori- 
ented programming 


Detailed interviews with 
key members of 18 large 
development projects to 
model their decision- 
making and communication 
process 
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Results of the Field Study 


• Observations about commonality/difference of projects 

j 

• Identification of five areas of organizational breakdown (within 
that sixteen specific problems) 

• Implications for process modeling 

• Mapping of problems onto lower-level phenomena 


"You need to understand, this project isn't the way we develop 
software at our company." 
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Characteristics of Projects Studied 


Pro- 

ject 

Stage of 
Life Cycle 


Characteristics 
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KLOC 

Real 

time 

hi 

111 

Gov. 

1 

Terminated 






Support Software 

2 

Development 

24 

✓ 


✓ 


Radio Control 

3 

Development 

50 

✓ 

✓ 

✓ 


Process Control 

4 

Development 

50 

✓ 




Operating System 

5 

Design 

70 





CAD 

6 

Development 

130 




✓ 

CAD 

7 

Development 

150+ 

✓ 


✓ 

✓ 

Avionics 

8 

Maintenance 

194 

✓ 



✓ 

C 3 

9 

Development 






Compiler 

10 

Maintenance 

250 





Run-time Library 

11 

Development 

350+ 





Compiler 

12 

Maintenance 

400 





Transaction Proc. 

13 

Design 

500 

✓ 

✓ 



Telephony 

14 

Maintenance 

725 





Operating System 

15 

Development 

1000 

V 

V 



Telephony 

16 

Maintenance 

50k+. 

✓ 

✓ 

✓ 

✓ 

Radar, C 3 

17 

Requirements 

100k+ 

✓ 

✓ 

✓ 

✓ 

C 3 , Life Support 
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Summary of Results from MCC Field Study* 


• Analysis of three significant problems 

• Layered behaviorial model of software processes 

• Conclusions and implications 


* Paper appearing in this months CACM 
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Analysis of Three Significant Problems in Software Design for Large Systems 




Layered Behavorial Model of Software Processes 
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Implications of Field Study Results 


• For Software Technology 

- Environment support needed for: 

= Knowledge integration 

= Change facilitation 

= Broad communication and coordination 

- Beginnings of an empirical model to measure improvement for a tool/practice 

• For Project Management 

- Expertise is the primary determinant, new ways of effectively organizing should be pursued 

- Key role players identified and described: 
superconceptualizer, diagnostician, gatekeeper, boundary spanner 

- Coordination by shared model of process, product 

• For Software Process Models 

- Difference between prescriptive and actual processes 

- Current process models do not reflect: 

learning, technical communication, requirements negotiation, and customer interaction 

- Framework for an "ideal" process model emerging 

• For Further Empirical Research on Professional Software Engineering 

- Much more to do 

- Focus on "variation" and its effect on the difference in productivity and quality outcomes 
among people, situations, and their interaction 
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The Software Project as an Ecological System 


Changing World 


Industry and Company 
P&P, Standards, Etc. 


Project 


Internal Processes of Interest 

• Assimilation of knowledge 

• Communication and coordination 

• Managing change 

• Issue resolution and decision making 

• Technical design 

• Organizational bureaucracy A 


Application 

Knowledge 


Contracting , 
Mechanisms 


Specific Needs 
of Customers 
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Five Crucial Problem Areas in Large Software Projects* 


External Context 

Change & 

I I Uncertainty 


Project 


Communication 


Application 

Knowledge 


Communication 
and Coordination 


Technology 

Transfer 


Design Evolution 
and Analysis 


see STP-390-86p 
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Overall Conclusion 


The Greatest Leverage Is in Supporting the Intersection of: 


The Technical Task 

/ 

• Assessing customer needs 

• Assimilating application knowledge 

• Negotiating requirements, technology, and resources 

• Identifying and exploring design assumptions/alternatives 

• Decomposing and recomposing functionality 

• Defining and controlling component interfaces 


The ManagementTask 

• Strategically managing system features and attributes 

• Assessing and controlling risks 

• Ensuring developers work from the same models 
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Results of the "LIFT" Study 



Observations on relative effort distribution 

j 

Observations about individual differences 
Identification of six process breakdowns 
A cognitive model of design problem solving 
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Information Model of Design Exploration 
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Individual Differences in Software Design Strategies 


Domain-Specific Strategies 


Exemplar driven 


Experienced 


Method (process) driven \ , ntermediate 


Computational paradigm driven 


Novice 


Trial and error driven 


Beginner 


General computation strategies 
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Results of the Team Design Study* 


• Identification of conflict behavior as key to achieving shared models 

• Observations on the limitations of "documents" 

• Observation of ombudsman to facilitate communication between 
customer and design teams 

• Observations on the effect of midnight prototype creation 

• Videotape identified as history capture mechanism 


* being completed at U.T. - D. Walz, 1988 
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Future SSEs Should Contain Facilities For 


1 ) Focus on Productivity and Quality 

- Statistical QC 

- Reduce waste and redundancy 

- Institutionalized reuse process yields component parts (via standards) 

2) Process Engineering 

- Introduction ot good practices, tools, etc. 

- Process definition, tailoring, monitoring, analysis, and improvement 

- Embodiment in education programs 

3) Process Efficiency through Teamwork and Communication 

- Revocation of Brook's Law 

- High performance teamwork 

- "Groupware" 

4) Flexible Organization Evolution 

- Coordinated technology, policy and organizational structure 
around process management concerns 

- Committment to improve (facilitation of change) 

- Capture of corporate domain knowledge (via issue-oriented domain analysis) 

- Negotiation-based requirements technology 

5) Liveware Support 

- Variety of "experts" (stakeholders) 

- Significant variation in abilities 
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Motivational Slide for this Morning 


In a study of 38 U.S. and Japanese Companies a wide variety of software management 
strategies were observed (Cusumano, 1 987). It was concluded that Japanese firms are 
significantly ahead in applying a disciplined and flexible factory approach, as evidenced by: 


Japan 

.26 bugs 
1000 SLOC 

5% projects 
late 

34% reuse 


U.S. 

8.3 bugs 

43% projects 

1 5% reuse 


1000 SLOC 

late 
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